Skip to content

Add testing of pg_upgrade - #31

Merged
jnasbyupgrade merged 9 commits into
masterfrom
phase4-pg-upgrade
Aug 7, 2026
Merged

Add testing of pg_upgrade#31
jnasbyupgrade merged 9 commits into
masterfrom
phase4-pg-upgrade

Conversation

@jnasbyupgrade

@jnasbyupgrade jnasbyupgrade commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Adds binary pg_upgrade coverage — proving count_nulls' objects survive a real PostgreSQL major-version upgrade, not just an in-place ALTER EXTENSION UPDATE. This can't live inside a plain pg_regress run since pg_upgrade is an external binary process.

What this does

New pg-upgrade-test CI job: install the oldest version we still ship a full script for on the oldest PostgreSQL major count_nulls supports, plant a dependency guard (so an accidental CASCADE drop anywhere in the job can't silently make this test a fresh install instead), binary pg_upgrade to the newest supported major, then run the suite against the real migrated database.

A single leg tests both real-world orderings a user could hit — updating the extension before upgrading PostgreSQL, and updating it after — via twin databases migrated by the same pg_upgrade call, so it only costs one binary upgrade instead of two. See the job's own comment for why one PostgreSQL floor is enough to cover both orderings.

bin/test_existing factors out the pieces that are genuinely external to pg_regress (preparing the old cluster, running the suite via --use-existing against the real upgraded database afterward) into a small, reusable script rather than inline CI YAML — modeled on the equivalent script in Postgres-Extensions/cat_tools.

Not yet crossed with TEST_SCHEMA.

Verification

Real CI run confirms the full flow end to end: both twin-database orderings, the binary pg_upgrade call itself, and the suite passing against the migrated objects. Locally (this container's own PostgreSQL clusters are persistent shared infra, not disposable, so the actual binary pg_upgrade step is CI-only): prepare-oldupdaterun-suite all pass standalone.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 637f74ab-65df-4b71-8455-49fb02b39fa9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@jnasbyupgrade jnasbyupgrade changed the title Phase 4: real pg_upgrade support via a reduced bin/test_existing Real pg_upgrade support via a reduced bin/test_existing Aug 4, 2026
@jnasbyupgrade
jnasbyupgrade marked this pull request as draft August 4, 2026 21:11
@jnasbyupgrade
jnasbyupgrade force-pushed the phase4-pg-upgrade branch 2 times, most recently from 6c1d36f to e64f80f Compare August 5, 2026 18:14
@jnasbyupgrade
jnasbyupgrade force-pushed the phase4-pg-upgrade branch 2 times, most recently from a9c46c2 to a791789 Compare August 5, 2026 22:49
@jnasbyupgrade
jnasbyupgrade force-pushed the phase4-pg-upgrade branch 2 times, most recently from 3b33ad8 to f37b039 Compare August 6, 2026 22:10
Base automatically changed from phase3.5-ci-hygiene to master August 6, 2026 23:52
@jnasbyupgrade
jnasbyupgrade marked this pull request as ready for review August 7, 2026 18:11
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Claude finished @jnasbyupgrade's task in 5m 47s —— View job


Code review

  • Gather context (PR body, changed files)
  • Review .github/workflows/ci.yml changes
  • Review bin/test_existing
  • Review bin/test_existing.sql/*.sql
  • Post review feedback

Overall this is a well-reasoned, carefully-commented PR — the two-leg matrix design (updating on opposite sides of the binary pg_upgrade) is a nice way to get both real-world orderings out of the same CI cost, and the dependency-guard self-check (plant → prove → re-prove → drop) is a solid way to make a silent CASCADE/reinstall fail loudly instead of passing green by accident. I read through ci.yml's new pg-upgrade-test job, bin/test_existing, and all three of its SQL helpers, and traced the step ordering (install-before-upgrade, update-before-vs-after-upgrade, guard survival checks) end to end — it's internally consistent and the two legs do test what the comments claim.

One real issue, otherwise minor:

bin/test_existing:125 — latent set -e trap in assert_version()

assert_version() {
  local db=$1 expected=$2 installed
  [ "$expected" = current ] && expected=$(current_version)
  ...

Under set -euo pipefail (line 56), && used as a conditional is dangerous: if $expected is ever anything other than the literal string current, the [ ] test is false, the whole command1 && command2 construct returns non-zero, and — because there's no enclosing if/||/similar to absorb that — the script aborts right there under set -e, with no FAIL: message at all.

This is exactly the failure mode this file's own author already flagged and avoided a few lines below, in update_ext() (line 138): "Use if, not &&: a false test under set -e would abort." The same fix wasn't applied here.

It's currently dormant — assert_version is only ever called as assert_version "$db" current (in run_suite, line 183) — so it can't manifest today. But the function's own signature (DB EXPECTED) implies a specific version is a supported input, and if a future caller ever passes one, the script would die silently instead of running the intended comparison. Worth the one-line fix for consistency with the pattern already established in this same file:

if [ "$expected" = current ]; then expected=$(current_version); fi

Fix this →

Nothing else stood out: the if: matrix.old_pg == '12' / '10' step gating matches the described leg semantics, the install-into-new-cluster-before-binary-pg_upgrade ordering is correct (the new cluster needs the control/update scripts on disk before pg_upgrade replays CREATE EXTENSION), the guard SQL's exception handling correctly rolls back the probe DROP EXTENSION even in the failure branch, and FLOOR=10/NEWEST=18 in the changes job confirms the "oldest supported version" claim in the old_pg=10 leg's comment.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Claude finished @jnasbyupgrade's task in 2m 20s —— View job


Code review

  • Gather context (PR body, changed files, prior review comments)
  • Check diff since last review (new commits from pull request synchronize)
  • Review .github/workflows/ci.yml changes
  • Review bin/test_existing
  • Review bin/test_existing.sql/*.sql
  • Post review feedback

This push (d714ef5, "confine the ordering comparison to old_pg=10 via twin databases") only touches .github/workflows/ci.ymlbin/test_existing and its SQL helpers are unchanged since the prior review. I traced the new twin-database flow end to end and it's internally consistent:

  • old_pg=10's two prepare-old calls (count_nulls_upgrade_oldest_first, count_nulls_upgrade_current_first, ci.yml:368-369) create both databases in the same old cluster before the single binary pg_upgrade call (ci.yml:409-415), which upgrades the whole cluster and therefore migrates both at once — matching the "one pg_upgrade call, twin databases" claim in the header comment.
  • The pre-upgrade update step (ci.yml:393) now runs unconditionally but selects the right target database via the matrix.old_pg == '10' && '...' || '...' GitHub Actions ternary — correctly touching only count_nulls_upgrade_current_first on that leg and leaving count_nulls_upgrade_oldest_first frozen at 0.9.6 for the post-upgrade update (ci.yml:438).
  • old_pg=12 cleanly reverts to the original single-database, single-ordering shape (count_nulls_upgrade, ci.yml:378/393/462), with no leftover if: matrix.old_pg == '12' guard on the now-unconditional pre-upgrade update step.
  • Both run-suite calls for the old_pg=10 leg (ci.yml:450-451) run sequentially in the same step against the two independently-migrated databases — each gets its own version/guard assertions.

No bugs found in this diff.

One item from the previous review remains open, since it's in a file this push didn't touch:

bin/test_existing:125 — latent set -e trap in assert_version()

[ "$expected" = current ] && expected=$(current_version)

Under set -euo pipefail (line 56), if $expected is ever anything other than the literal string current, this && short-circuits false and aborts the whole script with no FAIL: message — the exact pitfall this file's own update_ext() (line 138) already documents and avoids with an if. Still dormant today (assert_version is only ever called with "$db" current, run_suite:183), but worth the one-line fix for consistency:

if [ "$expected" = current ]; then expected=$(current_version); fi

Fix this →
· branch: phase4-pg-upgrade

jnasbyupgrade and others added 6 commits August 7, 2026 15:45
Adds the pg-upgrade-test CI job: install 0.9.6 on an old PostgreSQL major,
plant + prove a dependency guard, binary pg_upgrade to a newer major,
ALTER EXTENSION UPDATE the migrated objects, then run the suite against
the real upgraded database in existing mode.

bin/test_existing is much smaller than the equivalent script would have
been pre-test/install: only prepare-old and run-suite are genuinely
external-to-pg_regress concerns (a real pg_upgrade binary run isn't
something pg_regress can invoke itself), plus a small `update` subcommand
for the post-upgrade ALTER EXTENSION UPDATE step. There's no
update-scenario subcommand at all - that entire scenario is just `make
test-update` now (test/install/load.sql's own 'update' mode, added in
phase 3), since an in-place update has no external step to drive.

run_suite() gates on plain `make test`, not the old belt-and-suspenders
`make test && make verify-results` - pgxntool 2.3.0 (this repo's phase 0)
already made `make test` itself exit non-zero on regression failures.

Not yet crossed with TEST_SCHEMA - that's the next phase, once both this
job and extension-update-test can cross it together.

Verified locally against PG17 (prepare-old -> update -> run-suite, without
a real pg_upgrade - this container's clusters are persistent shared
infra, so the actual binary pg_upgrade leg is left for CI's ephemeral
containers, same reasoning as the pg-tle-test work).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… not after

Reorders prepare-old -> update -> pg_upgrade -> run-suite (was prepare-old
-> pg_upgrade -> update -> run-suite). The old order proved pg_upgrade
could migrate 0.9.6's frozen objects, then updated afterward - not
actionable, since that version already shipped. This job's whole point is
proving pg_upgrade correctly migrates the objects count_nulls' CURRENT code
creates, which requires updating BEFORE the binary upgrade runs. make
install (into the old cluster) already happens earlier in the job, so the
current version's update scripts are on disk in time for the moved step.

Updates the job's step names/comments and bin/test_existing's own
file-header sequence description to match the new order.
Propagates the draft-PR gating from phase3.5-ci-hygiene to the
pg-upgrade-test job introduced by this branch: same needs:[changes]/
if: docs_only pattern as pg-tle-test, so it gets the same
&& github.event.pull_request.draft != true guard.
old_pg=12 keeps updating to current before the binary pg_upgrade (proves
pg_upgrade preserves CURRENT-code objects); old_pg=10 now updates after
instead (proves pg_upgrade preserves objects still frozen at the oldest
supported version at upgrade time - a real user scenario the
update-before-only setup missed). Same 2-leg matrix, no added CI cost.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…n databases

The old_pg=10-vs-12 split wrongly conflated two independent axes (which PG
floor is tested, and which update-order is tested). Both orderings are now
compared on the SAME floor (old_pg=10, the oldest supported) using twin
databases migrated by a single pg_upgrade call - the same trick this file
already uses to cross TEST_SCHEMA without doubling matrix legs. old_pg=12
reverts to its original single-database, single-ordering design and is now
just a second, unrelated old-PG-floor data point.
…oded majors

old_pg=["10","12"] had no technical justification for the second leg (no
dependency-version SQL, no other package tie) and old_pg=10 stopped being
the real floor once #51 derived it as 9.4 from META.json. Add a floor_pg
output to the changes job (same $FLOOR already computed for supported_pg,
just re-emitted as a bare scalar like newest_pg) and drive both old_pg and
new_pg from those single sources of truth instead of separately hardcoded
literals. With only one leg left, the matrix.old_pg == '10'/'12'
conditionals that used to pick which twin-database ordering(s) to run no
longer make sense - both orderings now always run unconditionally.
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Claude finished @jnasbyupgrade's task in 2m 5s —— View job


Code review

  • Gather context (PR body, prior reviews, changed files)
  • Check diff since last review (commit 83b6214)
  • Review .github/workflows/ci.yml changes
  • Confirm CI results for the new leg
  • Post review feedback

This push (83b6214, "single META.json-derived floor for pg-upgrade-test, not two hardcoded majors") only touches .github/workflows/ci.ymlbin/test_existing and its SQL helpers are unchanged since the prior reviews.

The change collapses pg-upgrade-test's old_pg: ["10", "12"] matrix into a single leg driven by a new floor_pg output (ci.yml:107, 239) — the same $FLOOR value supported_pg already derives from META.json's declared minimum, re-emitted as a bare scalar exactly like the existing newest_pg pattern. I traced this through:

  • old_pg: ["${{ needs.changes.outputs.floor_pg }}"] / new_pg: ["${{ needs.changes.outputs.newest_pg }}"] (ci.yml:366-367) is the same single-element-list-from-scalar-output idiom already used for the test job's draft-PR reduction (ci.yml:283), so it's consistent with prior art in this file, not a new pattern to distrust.
  • All the now-dead if: matrix.old_pg == '10' / '12' conditionals were removed cleanly — confirmed no leftover references (grep -n "matrix.old_pg ==" returns nothing), and the two run-suite/update steps that used to be split per-leg are correctly merged into the twin-database flow that now runs unconditionally.
  • The reasoning for dropping the second leg holds up: the update-vs-pg_upgrade ordering comparison is a property of pg_upgrade itself, not of which floor it runs from, so a second floor was never adding coverage — it was just re-hardcoding a number that had already drifted from META.json's real 9.4 floor once CI: derive changes job's PG floor from META.json instead of a hardcoded 10 #51 landed.
  • This isn't just correct on paper — CI on this exact commit ran the new 🔄 Binary pg_upgrade 9.4 → 18 job and it passed, so the 9.4 floor is confirmed installable/upgradable in practice via the same pg-start/pg_createcluster machinery the 🐘 PostgreSQL 9.4 leg already exercises, not just a theoretical value.

No bugs found in this diff.

One item from the previous two reviews remains open, since it's in a file this push didn't touch:

bin/test_existing:125 — latent set -e trap in assert_version()

[ "$expected" = current ] && expected=$(current_version)

Under set -euo pipefail (line 56), if $expected is ever anything other than the literal string current, this && short-circuits false and aborts the whole script with no FAIL: message — the exact pitfall this file's own update_ext() (line 138) already documents and avoids with an if. Still dormant today (assert_version is only ever called with "$db" current, run_suite:183), but worth the one-line fix for consistency:

if [ "$expected" = current ]; then expected=$(current_version); fi

Fix this →
· branch: phase4-pg-upgrade

…g consumers

The comments explaining supported_pg/newest_pg/floor_pg (and the newest_pg
bare-scalar rationale) tied each output's reason for existing to a
specific downstream job/feature (test's draft-PR matrix reduction,
pg-upgrade-test's legs) and, for floor_pg, narrated the PR #51 history of
the bug it fixed. Rework them to just state what each value is and its
general purpose, so the comment doesn't need updating every time a new
consumer is added or removed.
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Claude finished @jnasbyupgrade's task in 1m 1s —— View job


Code review

  • Gather context (PR body, prior reviews, changed files)
  • Check diff since last review (new commit 937a7ac)
  • Review .github/workflows/ci.yml changes
  • Confirm status of previously-flagged assert_version() issue
  • Post review feedback

This push (937a7ac, "ci.yml: describe changes job outputs by current purpose, not by naming consumers") is comment-only — confirmed via git show 937a7ac, every added/removed line is a # comment in the changes job. It reframes the supported_pg/newest_pg/floor_pg output docs to describe what each value is rather than which specific downstream job consumes it (dropping references to test's draft-PR reduction, pg-upgrade-test's legs, and the PR #51 history), so the comment doesn't need editing every time a consumer is added or removed. No behavior changes: echo "supported_pg=..." / newest_pg=... / floor_pg=... and every step order/output name are byte-identical to the prior commit. bin/test_existing and its SQL helpers are unchanged since the prior reviews.

No bugs found in this diff.

One item from the previous three reviews remains open, since it's in a file this push didn't touch:

bin/test_existing:125 — latent set -e trap in assert_version()

[ "$expected" = current ] && expected=$(current_version)

Under set -euo pipefail (line 56), if $expected is ever anything other than the literal string current, this && short-circuits false and aborts the whole script with no FAIL: message — the exact pitfall this file's own update_ext() (line 138) already documents and avoids with an if. Still dormant today (assert_version is only ever called with "$db" current, run_suite:183), but worth the one-line fix for consistency:

if [ "$expected" = current ]; then expected=$(current_version); fi

Fix this →
· branch: phase4-pg-upgrade

…n db names

pg-start's own NO_CLUSTER option lets us install PostgreSQL and create the
old pg_upgrade-test cluster with --data-checksums in one step, instead of
starting a default cluster only to immediately stop/drop/recreate it.

Also drop the redundant count_nulls_ prefix from the twin database names
(upgrade_oldest_first/upgrade_current_first) - we're already inside the
count_nulls repo/test suite.
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Claude finished @jnasbyupgrade's task in 1m 28s —— View job


Code review

  • Gather context (PR body, prior reviews, changed files)
  • Check diff since last review (new commit 7cc1d1f)
  • Review .github/workflows/ci.yml changes
  • Confirm status of previously-flagged assert_version() issue
  • Post review feedback

This push (7cc1d1f, "skip the old cluster's create/destroy/recreate cycle, shorten twin db names") only touches .github/workflows/ci.ymlbin/test_existing and its SQL helpers are unchanged since the prior reviews. Two changes, both cosmetic/mechanical rather than behavioral:

  • Cluster setup: replaces pg-start <ver> (which starts a default cluster) + pg_ctlcluster stop + pg_dropcluster + pg_createcluster -p 5432 ... -- $INITDB_OPTS with a single NO_CLUSTER=1 pg-start <ver> (install only, no cluster) followed directly by the same pg_createcluster -p 5432 ... -- $INITDB_OPTS. This drops the redundant create→stop→drop→recreate cycle while ending at the same state: one cluster, on port 5432, with --data-checksums. pg-start is the pgxn/pgxn-tools container's own tool (ci.yml:281/365/476 container: pgxn/pgxn-tools), external to this repo, so I can't grep its source for NO_CLUSTER — but the "🔄 Binary pg_upgrade 9.4 → 18" job on this exact commit passed in CI (run 31221462721), confirming the flag behaves as described and the old cluster still comes up correctly with checksums enabled.
  • Naming: count_nulls_upgrade_oldest_first/count_nulls_upgrade_current_firstupgrade_oldest_first/upgrade_current_first, consistently across the header comment, the prepare-old/update/run-suite step bodies, and their inline comments. I checked for stragglers (grep -n "count_nulls_upgrade") — none left.

No bugs found in this diff, and no regressions: every job in the run, including all 🐘 PostgreSQL * legs, 🧩 pg_tle * legs, and the binary pg_upgrade leg, passed.

One item from the previous three reviews remains open, since it's in a file this push didn't touch:

bin/test_existing:125 — latent set -e trap in assert_version()

[ "$expected" = current ] && expected=$(current_version)

Under set -euo pipefail (line 56), if $expected is ever anything other than the literal string current, this && short-circuits false and aborts the whole script with no FAIL: message — the exact pitfall this file's own update_ext() (line 138) already documents and avoids with an if. Still dormant today (assert_version is only ever called with "$db" current, run_suite:183), but worth the one-line fix for consistency:

if [ "$expected" = current ]; then expected=$(current_version); fi

Fix this →
· branch: phase4-pg-upgrade

[ "$expected" = current ] && expected=$(current_version) aborts the
whole script silently whenever $expected isn't literally "current":
under `set -e`, a false left side of && makes the compound command's
exit status non-zero, which is fatal. Switch to `if`, matching the
same fix already applied to update_ext() a few lines below.
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Claude finished @jnasbyupgrade's task in 48s —— View job


Code review

  • Gather context (PR body, prior reviews, changed files)
  • Check diff since last review (new commit c094b68)
  • Review bin/test_existing fix for assert_version()
  • Post review feedback

This push (c094b68, "bin/test_existing: fix assert_version's false-test-under-set-e abort") is a one-line fix, touching only bin/test_existing:125:

-  [ "$expected" = current ] && expected=$(current_version)
+  if [ "$expected" = current ]; then expected=$(current_version); fi

This resolves the only issue raised across all five previous reviews of this PR. Confirmed:

  • It's the exact if-based rewrite recommended, and matches the sibling pattern already established a few lines below in update_ext() (bin/test_existing:139, with the comment there explicitly calling out the same set -e pitfall).
  • No other lines changed — git diff HEAD~1 HEAD shows a single insertion/deletion, confined to this file.
  • No behavioral change for existing callers: assert_version is still only ever invoked as assert_version "$db" current (run_suite, line ~183), so this doesn't alter today's control flow — it just removes the latent trap for any future caller that passes an explicit version.

No bugs found. All previously-flagged issues in this PR are now resolved.
· branch: phase4-pg-upgrade

@jnasbyupgrade jnasbyupgrade changed the title Real pg_upgrade support via a reduced bin/test_existing Add testing of pg_upgrade Aug 7, 2026
@jnasbyupgrade
jnasbyupgrade merged commit f578dc0 into master Aug 7, 2026
25 checks passed
@jnasbyupgrade
jnasbyupgrade deleted the phase4-pg-upgrade branch August 7, 2026 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant